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PREFACE 



This report is the fourth in a continuing series of cooperative 
ventures between the Naval Training Equipment Center and the Ain Force Human 
Resources Laboratory. This experiment was carried out on the Navy's Visual 
Techology Research Simulator (VTRS) in Orlando', Florida, and is the first in 
this cooperative series to employ the VTRS. 

A subject of concern to both the Navy and the Air Force was addressed 
in the experiment reported here. Issues concerning the implementation of 
instructional strategies and employment , of certain simulator features in 
line with their instructional value have been addressed in the recent past. 
The issue addressed in the experiment reported here concerns the 
instructional value of one »common simulator feature, the "freeze" feature. 
The "freeze" feature allows for a suspension of the simulator task so that a 
student may be given instructional feedback while freed of the requirement 
to perform the task* The experiment reported here concerns the "freeze" 
feature commonly found in flight simulators and whether it should be 
employed in the training of a complex flight task. — ^ 

A number of persons contributed to'-this research. Walter S. Chambers, ^ 
, Stanley C. Collyer, Patricia Daoust and Edward Holler of the Naval Training 
Equipment Center (Code N-732f; Brian Nelson, Daniel ySheppard and Daniel 
Westra of Canyon Research Group, Inc.; and Jack Davis and Karen Thomley of 
tjie University of Central Florida provided technical support. 

Twenty-five Air Force pilots from the 56th Tactical Fighter Wing, 
Mac^ill Air Force Base, Florida served as subjecys in this experiment. They 
feire to be commended for their cooperation. The authors would like to 
specifically thank Major CP. Dockery of the 9ti/ Air Force, Shaw Air Force 
Base, South Carolina and Col. D. McCarcer, the /Naval Training Equipment 
Center\s Air Force Liaison officer for their v^aluable assistance in securing 
the pilots who participated. / 
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SECTION I 
INTRODUCTION 



The history of simulation has closely followed the history of 
aviation. Early trainers were crude* unsophisticated boxes designed to give 
pilots some semblance of the flying experience and to prepare them to 
perform procedures important in flight. With the passage of time, aircraft 
and flight simulators have undergone evolutionary changes. Modern flight 
simulators are capable of reproducing many of the same conditions that are 
attainable in the aircraft (Caro, 1977). This refinement, in simulator 
capability is an outgrowth of advances in technology and is due to an 
increased desire on the part of the simulator design engineer to give the 
pilot an environment that '"feels" as much as possible like the real , 
aircraft. Pilot self-report and subjective assessment were the ruling 
criteria that drove the development of flight simulators. 

« 

Thus the flight simulator has been regarded as an artificial airplane 
wherein flight tasks could be performed in a safer, cheaper environment. 
Economy and safety were the specific reasons to substitute flight simulators 
for aircraft. However recent advances in training technology have increased 
awareness that the atf^raft may not be the best place to begin the process 
of learning to fly. Tnfexsimulator, in contrast to the airplane, has the 
potential to be structured as a learning environment that can facilitate the 
acquisition of the perceptual and motor skills necessary for aircraft 
control (Caro, 1976). 

Reliance upon an "in-flight" model to dictate the limits of design and 
use of simulators for pilot training can unnecessarily restrict the 
potential of simulation to facilitate skill acquisition* Recent research 
has begun to address the issue of how to enhance the training value of 
simulation by employing principles of learning and transfer rather than 
sheer physical fidelity as the guiding criteria (Hughes, 1979) j The 
employment of certain techniques that follow from these principles may lead 
to simulator conditions that, although objectively unrealistic, can enhance 
the efficiency of the simulator as a training device. 

For example, Hughes, Hannan, and Jones (1979) have explored the use of 
a record-playback feature for instruction of a complex flight task while 
Bailey, Hughes,* and Jones (1980) have tested the instructional value of 
backward chaining with a dive bombing task. Lintern (1980) and Hennessy, 
Lintern, and Col Iyer (1981) have examined learning with novel and altered 
visual displays." 

# 

Simulators not only offer the promise of permitting learning to proceed 
more rapidly, but also may permit a higher level of skill attainment. With 
some specific flight tasks, the benign and predictable simulation 
environment may permit pilots to quickly perceive critical relationships 
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that can aid their performance in the aircraft but that would never clearly 
emerge if they practiced only in the aircraft. 

These studies represent initial attempts to explore the potential of 
flight simulators as training devices through the alteration of the display 
presentation or through the application of principles of learning as 
training features. This point of view diverges from the artificial aircraft 
model and requires that relevant learning principles be addressed as the 
driving force behind the conduct of training in iflight simulators. 

Of all the instructional options that have been, or might be, built 
into an aircraft simulator, "freeze" appears to have gained the widest 
acceptance. "Freeze," in th.e simulator training context, refers to the 
suspension of any part or all of the simulated task for instructional 
purposes. Two uses may be made of this "freeze" feature. First, "freeze" 
can be employed to suspend some aspect of the flight dynamics (e.g., 
aircraft roll may be "frozen") so that the trainee may focus ,h is attention 
upon some other, more critical aspect of the task, prior to attempting the 
whole task. Another potential use of "freeze" that has high apparent 
validity is for the instructor to stop the action while explaining errors 
and strategies to the student. Nevertheless, there must be some concern 
that the level of assistance provided by the use of "freeze," especially in 
the latter case, could disrupt retention of the skill (c.f.. Snow, 1980) or 
that the interruption could disrupt its acquisition. 

The present study is concerned with the use of the latter, total task 
suspension "freeze" technique, and how it relates to the specific role of 
errors in skill acquisition; in particular, whether a "freeze" should be 
used as an opportunity to instruct the student in the cause and correction 
of errors or whether it should be used to minimize them. While it is 
apparent that information about the direction and magnitude of errors can 
facilitate skill learning (Ad.ams, 1981), there is some concern that . 
frequently committed errors could become embedded in a student's response 
repertoire (Holding^ 1970). Thus approaches that enhance the student's 
awareness of the nature of the error and those that minimize errors 
represent extreme positions in the treatment of errors during learning. 

The possibility that repetition will embed errors in a student's 
behavior appears possible in the* case of the carrier landing task. For 
example, a marginal approach, while being severely criticized by the Landing 
Signal Officer (LSO), can result in a safe landing. Although Navy pilots 
are consciously and explicitly concerned with technique in making carrier 
approaches, successful completion of a dangerous task by any means is, in a 
technical sense, reinforced. This positive reinforcement may partially 
negate the effects of negative reinforcement from 'the LSO and from the 
pilot's own cognitive judgments and could be one of the more potent 
influences on learning. The natural consequences of errors app'fear to have a 
powerful self-correcting influence in perceptual-motor learning, but the 
natural consequences of errors in carrier approaches may not be sufficiently 
negative with the required frequency to give full force to this effect. 
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Given the expanded range of options available for simulator instruction 
versus in-flight instruction, the questions arise as to whether errors 
should be permitted, how they should be treated if they are permitted, and 
how an instructor might intervene to enhance the learning process. The 
carrier landing task was considered ideal for examining these issues. 
Control behavior is strictly constrained by the demands of the task, and 
errors can be specified clearly. Navy pilots and instructors tend to agree 
that error detection and correction are fundamental to safe and consistent 
carrier landings. 

The present study addressed the issue of how errors should be treated 
by examining three techniques for teaching the carrier approach task. 
Specifically, the study addressed the use of the simulator's "freeze" 
feature to interrupt an otherwise continuous performance whenever an error 
was detected. One possible advantage of freezing the task in this manner is 
that it would allow students to attend to instructional feedback without the 
simultaneous need for them to perform the task. Effectiveness of the 
"feedback might thus be enhanced and so lead to faster learning* 
Alternatively, interruption of the continuous task might be disruptive and 
thereby impede learning. The following instructional conditions were 
created to explore these issues. 

"Freeze/Reset Under the Freeze/Reset condition, the simulator was 
"frozen" whenever an error was detected. Feedback was given while the 
"freeze" was in effect. During the "freeze," inside- and outside-cockpit 
references and cues weVe maintainea as they were when the "freeze" 
occurred. Before continuing the task, however, the simulator was returned 
to the appropriate position (vertically) on glideslope with appropriately 
configured angle of attack and airspeed. At the termination of the 
"freeze," the student continued the task from the "corrected" position. 

"Freeze/Flyout " Under the Freeze/Flyout condition, the actions during 
the "freeze" were the same^^as those for the Freeze/Reset condition except 
that at termination of the "freeze," the student continued the task from the 
exact point at which it was "frozen." That is, no correction was made. 

"Conventional " Under the Conventional condition, students learned the 
task without use of the "freeze." Instructional feedback was given at the 
completion of each approach/ 

In addition to e'xamthing these three alternative instructional 
conditions, the study addressed the manner in which errors were defined. 
One error criterion was based on displacement from the glideslope while the 
other was based on both displacement from glideslope and deviation from the 
optimum rate of descent. The latter criterion is potentially more 
informative in that a descent-rate error can give advance warning of a 
displacement error so that earlier corrective action can be taken. The 
addition of rate information to displacement information has been shown to 
minimize glideslope tracking errors of experienced Navy pilots (Kaul, 
Col Iyer, and Lintern, 1980). 
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One important methodological issue in transfer-of-training (TOT) 
research is related to selection of an appropriate training period/ 
Training to a proficiency criterion has often been used, but in a study of 
differential transfer, the average time for various groups to attain 
proficiency almost alwavs differs. Thus, training time tends to be 
confounded with the ex^rimental effects of interest. Fixed training times 
resolve that problem but selection of an appropriate period can be critical, 
and necessarily relies heavily on the judgment and experience of the 
experimenter. Training times could be too short to allow differences to 
emerge. Alternatively, they could be so long that worthwhile training 
differences are washed out by subjects attaining a high level of proficiency 
with even the poorest training conditions. Thus, training times should be 
extended into, but not beyond, that period in training which shows 
worthwhile learning differences between instructional methods. 

Pre-experimental work could ascertain the most appropriate training 
period, but it would require expenditure of a large portion of the 
experimental resources to obtain a 'reliable answer. Furthermore, in a study 
of more than two training conditions, the selected training time may be 
appropriate for only some of the comparisons. A range of times could be • 
used but would, reduce the^power of the experiment (i.e., its capability to 
reveal differences between conditions) to the extent that some of the 
selected training periods were inappropriate. Training time is a special 
issue in a study of novel training techniques, such as those considered here 
where an experimenter has limited experience and meager data to provide 
guidance. 

A probe technique in which learning trials or the experimental 
conditions are interspersed with test trials on the control condition could 
avoid these problems. This technique, which appears to have been used only 
once in applied transf er-of-training research (Smith, Pence, Queen and 
Wulfetk, 1974) might effectively map the course of learning and thus allow 
an estimate of the optimum training period for each instructional method. 
Smithy al. (1974) used a single-trial probe strategy in which training and 
probe triTTs alternated. Their strategy was probably not optimum. 
Presumably an experimental session should be weighted heavily with training 
versus probe trials to limit dilution of the training effects. Nevertheless 
probes should be frequent enough to ensure that critical differences are not 
missed, and sufficient data would be required at each probe to achieve 
worthwhile stability. Probe methodology would seem to offer distinct 
advantages for the initial investigation of a novel training method. 
However, for evaluating savings in relation to'a standard instructional 
paradigm, the traditional transf er-of-training paradigm would still be 
preferred. In this experiment, the probe technique was employed in an 
attempt to solve some- of the dilemmas faced by transfer of training 
experimenters. Here probe trials, consisting of performance of the transfer 
task, were interspersed throughout the training sessions in order to assess 
the level of proficiency of the trainees as training progressed. Two 
critical issues surface as a consequence of the use of a probe technique. . 
First, are the probe performances sensitive to the learning that is taking 
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place during the training trials? Secondly, are the probe trials Jn some 
way disruptive of the training process? Both of these issues were 
addressed in the experiment. 

SUMMARY OF PRIMARY OBJECTIVES 

1. To assess the relative effectiveness of three instructional 
methods differing in the degree to which each altefs'the 
instructional environment following an error. 

2. To explore the total task suspensjon use of "freeze" for 
instruction of a continuous tracking skill. 

3. To examine the effects on learning of two error criteria: ' 
displaicement only versus .displacement and rate. 

4. To assess, the extent to which the transfer task performance 
sampled periodicaTly in probef trials is sensitive to what is 
learned in training trials and what effect, if any, probe trials 
might have upon learning. 
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SECTION II 
METHOD 



Five groups of five experienced Air Force pilots were taught carrier 
landings in a flight simulator at the Naval Training Equipment Center under 
a control or one of four experimental training conditions. 

Experienced Air Force pilots were sought to avoid the necessity of 
teaching basic aircraft control skills. These pilots were, however, 
inexperienced with reference to the specific carrier approach skills that 
were to be learned, in this experiment. Table 1 summarizes the flight 
experience of the pilots. 

APPARATUS 

The Visual Technology Research Simulator (VTRS) consists of a fully 
instrumented T-2C navy jet trainer cockpit, a si x-degree-of -freedom 
synergistic motion platform, a 32-element G-seat, a wide-angle visual system 
that can project both computer-generated and model-board images, and an 
Experimenter/Operator Control Station (Collyer and Chambers, 1978). The 
'motion system, G-seat, and model board were not used in this experiment. 

VISUAL SYSTEM. 'The background subtended 50' above to 30' below the pilot's 
eye level, and 80** to either side of 'the cockpit. The aircraft carrier 
image, which was a representation of the Forrestal (CVA 59) was generated by 
computer and projected onto the background through a 1025-line video 
system. A carrier wake and Fresnel Lens Optical Landing System (FLOLS) were 
also generated by this method. Both daytime and nighttime carrier images 
could be displayed (Figures 1 and 2). 

Average delay between control inputs and generation of the 
corresponding visual scene was approximately 116 msec (calculation of new 
aircraft coordinates required 50 msec while calculation of the visual scene 
corresponding to the viewpoint from the. new aircraft coordinates required 
approximately 5D msec and' generation of the new scene required 17 msec). An 
updated aircraft position was computed every 33 msec, but the picture 
position was .updated every 17 msec by extrapolating aircraft position in 
between each computed aircraft position. 

The sky brightness for the day scene was 0.85 fL (foot-lambert) and the 
seascape brightness was 0.6^ fL. The brightest area of the day carrier was 
4.0 fL. Except for the horizon,, no features were represented in either the 
sky- o.r sea. The night background luminance was 0.04 fL and the horizon and 
seascape were not visible. The night carrier appeared as lights of 0.8 fL n 
brightness outlining the landing deck and other features, ^ ' 
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TABLE 1. BIOGRAPHICAL DATA ON PILOT SUBJECTS 



Subject 



10 11 12 13 



' Assignment ' ^"^^ ^"^^ 

Age 35 35 27 32 33 32 29 37 32 31 33 34 32 

Aircrafr^' ^^^^ ^^^^ ^^^^ ^^^^ 

Simulator 10 470 550 360 300 350 300 450 400 194 210 90 300 

Last 30 Days 15/ 20/ 22/ 20/ 20/ 20/ 17/ 20/ 22/ 12/ ' 20/ 24/ 20/ 

Hrs. Aircraft F- 16 F-16 F-4D F-16 F-4 F-4D F-4D F-4 F-4D F-16 F-4 F-16 F-16 

' continued 



Subject 



Group 
Assignment 



Age 



TABLE 1, BIOGRAPHICAL DATA ON PILOT SUBJECTS (Cont'd.) 

14 15 16 17 18 19 20 21 22 23 24 25 

(5) (2) (1) (1) (4) (2) (5) (4) (2) (5) (1) (3) 

37 ,38 45 47 43 31 32 36 42 35 39 30 



Ai>?rafr^* ^^^^ 2°^^ 1^°° 2^°° 1^^° 

Simulator 215 170 150 500 295 200 210 300 350 390 260 250 

Last 30 Days 20/ 25/ ^ 15/ 5/ . 20/ 18/ ^ 15/ 25/ 25/ * 14/ 

Hrs. Aircraft F-4 F-4 F-4 F-4 F-4 F-16 ^ F-4D F-4D F-4 F-4D 



Key to Group Assignment 



1) Freeze/Flyout Display 

2) Freeze Reset/Displacement 

3) Conventional Display 

4) Freeze/Flyout Displacement and Rate 

5) Freeze Reset/Displacement and Rate 
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FRESNEL LENS OPTICAL LANDING SYSTEM. The configuration of the FLOLS is 
shown in Figure 3. In contrast to a carrier^ FLOLS, which is generated by 
incandescent lights and can ffierefore be much brighter than other parts of 
the carrier, the simulated FLOLS was generated by the same system as the 
carrier image. It was therefore only as bright as the brightest areas of 
the ship (e.g., the white lines on the landing deck). To compensate for its 
lower relative brightness, the FLOLS was enlarged by a factor of 4.5 when 
the distance behind the ramp was greater than 2250 ft. From 2250 ft., the 
size of the FLOLS was linearly reduced until it attained 1.5X its normal 
size at 750 ft. It remained that size throughout the remainder of the 
approach. The FLOLS was centered 414 ft. down the landing deck and 61 ft. 
to the left of the centerline. It was set at a nominal 3.5** glideslope and 
with a lateral viewing wedge of 52\ 

SIMULATOR CONFIGURATION. The simulator was initialized with the aircraft at 
9000 ft. from the ramp, on the glideslope and centerline, and in the 
approach attitude and configuration (hook and wheels down, speed brake out, 
15 units angle-of-attack (AOA), and power at 83%). The T-2C is normally 
landed with full flaps, but flaps were at half extension for this experiment 
to more closely simulate approach speeds of typical fleet aircraft. Fuel 
was set at 1320 lbs. to give 10,000 lbs. gross weight. A landing trial was 
flown from the initial condition to wire arrestment or, in the case of a 
bolter, to 1000 ft. past the carrier. 

The carrier was set on a heading of 360** at 5 knots. Environmental 
w.nd was set at 349. 5** with a velocity of 20.1 knots. This combination of 
carrier speed and environmental wind produced a relative component of 25 
knots down the landing deck. 

Turbulence was used to increase the difficulty. The turbulence model 
buffeted the simulator computed aircraft model with a random forcing 
function. 

PERFORMANCE MEASUREMENT. VTRS has the capacity to assess performance at 
30 Hz for three classes of measure. First, aircraft position can be 
determined throughout the approach. Secondly, aircraft control surface 
activity can be measured throughout the approach for elevator, aileron and 
rudder displacement. Finally, aircraft control position data can also be 
collected in order to determine throttle, control stick and rudder pedal 
activity throughout the approach. 

For each of these categories of measure, specific measures such as Root 
Mean Square (RMS) error, absolute error, and percent time within a 
predefined tolerance band may be collected throughout the course of an 
approach. In addition,, these measures can also be collected for any 
predefined segment (or segments) of the approach (e.g.^ 1500 ft. to 
touchdown). 
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Figure 3. Configuration of FLOLS Simulation, Showing Datum Bars and Meatball 
(Dimensions Shown are in Ft.). 
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Aside from continuous measures of glideslope performance, up to two 
separate snapshot measures may be collected at any point on the approach. 
This snapshot can contain any of the classes of measures deemed important. 

The capacity also exists for collecting touchdown performance data from 
which a measure of success of the landing may be calculated. 

FREEZE AND RESET FEATURES. The simulator has the capacity to suspend the 
ongoing simulation either manually, by the use of the "freeze" button 
located on the console, or the "freeze" feature may be instigated 
automatically based upon a specified error being committed by the trainee. 
Regardless of how the "freeze" is initiated, three actions are possible 
following the suspension of the simulation. First, the simulator may be 
reset to a corrected initialization point and the trainee can beqin to fly 
from that point. Secondly, the current simulator state may be restored and 
the trainee can continue the task from that point. Finally, the simulator 
may^^e set to some other, entirely different, initial condition and the 
trairl^e can be required to perform some other task. The first two of these 
post "freeze" actions were employed here as experimental training 
conditions. These training conditions and the criteria for defining an 
error are described below. 

TRAINING CONDITIONS. Two experimental training procedures were used in the 
experiment. For both procedures the simulator was frozen during the 
approach if the pilot's vertical deviations from the glideslope exceeded 
specific criteria. Under one procedure, known as Freeze/Reset, when the 
simulator was frozen, the pilots were advised on how they had incurred their 
vertical error and were then reset to the glideslope with the simulator in 
its optimum approach attitude. Longitudinal distance from the carrier and 
lateral distance from the extended centerline of the landing deck were not 
changed. Pilots continued their approach from the reset position. Under 
the other procedure known as Freeze/Flyout, pilots were advised on how th<»y 
had incurred their vertical error and how to correct it once they were 
released. They then continued their approach from the position and attitude 
in which the simulator had been frozen. 

Two experimental training conditions were derived for each Freeze 
procedure by applying tv/o different criteria. 

DISPLACEMENT ERROR CRITERION. This criterion "froze" the system if 

^i > ©c (1) 

,where = angular displacement of the aircraft from the 

3.5"" glideslope, 

©c = 0.5625 -r(0.3125 x 10-^), 0 £ r < 6000 

and 

r = range in feet from the carrier ramp. 

This algorithm linearly increased the criterion in meatball units from 1.0 
at 6000 feet from the ramp to 1.5 at the ramp. "Freezes" did not occur 
beyond 6000 feet from the carrier or past the ramp. 
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DISPLACEMENT AND DESCENT RATE ERROR CRITERION. The second error criterion 
would result in a "freeze" if vertical deviation from the 3.5° glideslope, 
descent rate error, or some combination of the two was excessive. "Freezes" 
would occur: 

1f ■ I Mil > Gc ■ (2) 

. -1 

for Mi = Gi + 0.5625 Gc 



= angular rate of displacement in degrees/second 
from the glideslope, 

Oc = 0.405 -(0.49 X 10-4) (r + tk) , 

and r^ = 524 feet, the dista'nce from the carrier ramp to 

the FLOLS origin. 

This algorithm established a criterion that was a weighted sum of the 
previously described displacement criterion and a descent rate error limit 
that decreased linearly from 600 fpm at 6000 feet from the ramp to 200 fpm 
at the ramp, 

"Freezes" were not permitted within 10 seconds of restarting the 
.approach after a previous "freeze." In addition, a "freeze" would not occur 
if, at the end of this 10-secbhd period, the subject was outside of the 
performance criterion but was, decreasing the error. 

In the fifth training condition, designated the Conventional, the 
simulator was not "frozen" during the approach but the subjects were given 
feedback as to their error (equivalent to that given the Freeze/Flyou.t. 
group) at the end of each trial. 

PROCEDURE 

Two subjects arrived at the simulation facility each day, Monday 
through Thursday, during the experiment.^ They viewed a vi&ep tape on 
carrier landings which described the FLOLS and carrier landirtqs. They were 
then given detailed instructions by ^a Navy LSO on carrier lanadng 
techniques. This instructional period Tasted approximately 45Viinutes. 
When convenient,. subjects were given this preliminary instructipn in pairs, 
but the remaining experimental work was undertaken with only on^ subject in 
attendance 'fexcept that subjects were occasionally permitted to monitor the 
performance of others from outside the simulator if they had entirely 
completed their experimental work. Subjects were assigned to training 
Conditions as they arrived at the simulator facility in accordance with a 
predetermined sequence that ensured the number of subjects having been 
trained' with each condition remained approximately equal throughout the 
experiment. 
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After preliminary instruction, subjects were familiarized with the 
controls of the simulator. They were then given a brief flight of 
approximately two minutes before they commenced their carrier landing 
training. The training sequence consisted of 24 approaches to the day 
ca»*rier.on the afternoon of their first day at the simulator facility, and 
24 approaches to the night carrier on the morning of the second day. The 
two 24-trial. blocks were divided into six-trial sub-blocks, the first four 
trials of whtth were flown under the appropriate training condition. The 
last two trials of each sub-block were used as probe trials to assess the 
progress of learning, and were flown under the control condition. The LSO 
gave no instructions during or following probe trials. Subjects were given 
a ten minute rest after the twelfth trial of each 24-trial block. 

PERFORMANCE MEASUREMENT AND DATA ANALYSIS 

Position, attixude, and state of the simulated aircraft and elevator, 
throttle, aileron and rudder control positions were sampled at 30 Hz. The 
simulator position, attitude, and state variables were used to derive 
performance measures for glideslope and AOA tracking. The control position 
variables were used to derive control activity measures: The continuous 
measures were derived for non-o.verlapping 1500-foot segments of the 
approach. 

Repeated measures analyses of variance were applied to root mean square 
(RMS) glideslope and RMS AOA errors and to flight control activity 
measures. In addition, multiple discriminant analyses were applied to the 
probe data to find linear combinations of scores that maximally 
discriminated the instructional treatments. 
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SECTION III 
RESULTS 



TRAINING TRIALS 

LEARNING EFFECTS. Several learning trends are apparent and show up as 
reductions in RMS error for glideslope (See Appendix A, Table Al) and AOA 
(Table A2), and as reductions in activity of elevator (Table A3) and 
ailerons (Table A5). There was comparable reduction in throttle activity 
(Table A4), while average rudder pedal activity (Table A6) showed no 
reliable effects. Figure 4 shows a consistent trend towards reduction in 
number of "freezes" (with a reversal in transitioning from Day to Night 
approaches) in the experimental conditions. This also indicates a steady 
improvement in glideslope control throughout the experiment. 

EXPERIMENTAL TREATMENT EFFECTS. Only RMS glideslope error scores showed any 
statistically reliable effect of experimental training conditionrs (Table 
Al). Glideslope errors of the Freeze/Reset groups were lower than for other 
groups , an unsurprising result in view of the fact that errors were 
constrained and frequently zeroed by the experimental manipulation. This 
suggests- that "freezes^* occurred frequently enough to reduce error and 
thereby validates the experimental manipulation to some extent. 

PROBE TRfALS 

LEARNING EFFECTS. kJhile error scores and control activity tended to 
decrease throughout probe trials, these trends were generally reliable only 
for the final, segment (last .1500 feet) of the Day approaches (session 1) 
with measures of RMS glideslope error (Table A7), RMS AOA error (Table A8), 
and elevator and aileron control activity (Tables A9 and AH). In contrast 
to other measures, the reductions in aileron control activity were 
statistically reliable over all four 1500-foot approach segments for both 
Day, and Night conditions. RMS glideslope error and aileron control activity 
showed a sharp increment in the transition from Day to Night approaches 
(which coincided with the break betwee/i sessions 1 and 2). 

EXPERIMENTAL TREATMENT EFFECTS. -Univariate analyses showed no reliable 
instructional treatment effects between probe-trial measures of RMS 
glideslope error, RMS AOA error and control activity. Tables A7 to A12 
summarize the tests of statistical reliability for these measures. Thus 
neither instructional technique nor the method of determining error appeared 
to have any clear effect on learning. 

In a further effort to seek a relationship between instructional 
techniques and learning, stepwise discriminant analyses were applied to 
the' six measures of Tables A7 to A12. The analyses were restricted to the 
data of the final approach segment (1500 to 0 feet) because they showed the 
most consistent learning trends. Data from the first pair of probe trials 
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were not included because they did not appear to follow the pattern of later 
probe trials, and because it was considered that differences due to learning 
might not have become established at that point. Pairs of probe trials were 
used to define dependent measures so that the numbers of dependent measures 
entered into the discriminant analyses were 18 (three probe pairs by six 
measures) for the day analysis and 24 (four probe pairs by six measures) for 
the night analysis. Day and night trials were analyzed separately to 
improve the ratio of subjects to dependent measures. Nevertheless, the 
analyses did not conform to the normal constraints (the size of the smallest 
group should exceed the number of variables, the total number of subjects 
should be two to three times the number of dependent measures (fatsuoka, 
1970)) and should be considered as purely exploratory. 

The cumulative proportion of total dispersion accounted for by the 
discriminant function of the Day data was 86% and the average correct 
classification of subjects into groups (using the jackknifed procedure) was 
36% (expected chance value of 20%). The overall approximate F ratio was 
statistically reliable at p<.01. Aileron control activity for the third and 
fourth probe trials contributed most to the separation of the groups on the 
first canonical variable. The trend was towards smoother aileron control 
inputs for pilots in the Conventional (no-*'freeze") condition. 

The cumulative proportion of the total dispersion accounted for by the 
discriminant function of the Night data was 94% and the average correct, 
classification of subjects into groups was 68%. The overall approximate F 
ratio was statistically reliable at p<.01. Pedal control activity at the 
sixth and seventh probe pairs, throttle control activity at the fifth probe 
pair and aileron, control activity at the sixth probe pair contributed most 
to separation of the grpups oh the first canonical variable. The trend was 
towards smoother throttle and aileron control inputs for pilots in the 
Conventional group, and for higher fhr^ottle activity for both Freeze. groups 
that used the Displacement-only criterion. 

PROBE METHODOLOGY 

Means of RMS glideslope errors and of aileron control activity in the 
final 1500 feet of the approach were examined for disruptive effects of 
transitions between tr&ining and probe trials (Figures 5 and 6). This 
segment was chosen for analysis because it showed the strongest and most 
consistent learning trends in both training and probe trials. The 
glideslope measure was examined because the experimental manipulations were 
intended to impact glideslope tracking. Aileron control activity was also 
examined because of the strong and consistent learning effects that were 
demonstrated with this measure. 

There v/ere no consistent differences between probe trials for the 
experimental groups and the trials for the Conventional groups that were in 
the probe locations. Nor were there consistent differences between first 
and second probe trials, between first and second training trials following 
a probe, or between training trials prior to and subsequent to a probe, that 
would suggest any disruption resulting from transitions between training and 
probe trials. 
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Figure 6. Means of Aileron Control Activity. 
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QUESTIONNAIRE RESPONSES . 

Descriptive data based on pilot responses to the paper-and-pencil 
questionnaire are given by individual item in Appendix B. The results are 
summarized below. . • 

4. 

ON THE GENERAL ROLE .OF ERRORS IN TRAINING. Pilots general ly .disagreed that 
"errors served little purpose" and also disagreed with the notion that 
"students may actually learn the errors they commit" (Item 12). Pilots also 
disagreed with the contention that "instructional methods that allow errors 
to occur are inefficient" (Item 14). Instead, pilots in tire study pointed 
to error recognition as a basis for the development of correct performance 
(Item 18). Errors were seen as helping the student to focus on the critical 
elements of task performance (Item 13), as well as exposing the student to 
out-of-tolerance situations which may, under later conditions, result from 
factors such as adverse weather, visibility/ceiling limitations, etc. (Item 
15). On the issue of whether correct performance is best thought of as 
resulting from a process of eliminating errors or from a process of shaping 
desired responses, pilots were undecided (Item 17). 

ON THE INSTRUCTIONAL USE OF THE FREEZE FEATURE. Pilots agreed that it was 
easier to attend to the LSO's feedback while the simulator was "frozen" than 
while trying to listen and fly the aircraft at the same time (Item 3)* 
Pilots also agreed that use of the "freeze" aided development of error 
recognition (Item 5) but were undecided as to whether it might significantly 
decrease the overall time required to learn the landing task. On the 
negative side, pilots indi.cated that the occurrence of the "freeze" early In 
training was "frustrating"' (Item 8). In fact, pilots in the Freeze/Reset 
condition Indicated that they were more motivated by "trying to avoid the 
"freeze" than by trying to fly the task correctly" (Item 7). Pilots in the 
Freeze/Flyout conditio^ tended to respond in somewhat the opposite manner. 
Regardless of the "freeze" condition to which subjects .were assigned, all 
indicated that regaining control of the simulator following a "freeze" 
significantly added to the difficulty of the flying task (Item 2) and that 
the difficulty increased the closer the occurrence of the "freeze" to the 
terminal portion of the task (Item 2)» 

In general, the questionnaire data indicated that pilots perceived 
errors as contributing positively to training, that the present use of the 
"freeze" feature was in some instances avers ive, and that it served to add 
to the difficulty of learning the task in the simulator despite the fact 
that the _||freeze" made it easier to attend to feedback from the LSO. So far 
as being able to potentially reduce the time needed to learn the task, 
pilots perceived the present application of the "freeze" to have little 
value. 
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SECTION IV 
DISCUSSION 



Learning appeared to be limited to the final 1500 feet of the approach; 
an observation that suggests that the experienced Air Forte pilots used in 
the experiment were able to perform the early part of the task with 
strategies similar to those used in normal landings. Only in the last part 
of the approach, where the altitude tolerances became very small, did the 
pilots appear to develop new techniques. This probably reduced the 
effectiveness of the expermental manipulation since it appears to have 
offered limited opportunity to "freeze," and to process errors subsequently 
during the "freeze," in that part of the approach that required new 
learning. Thus, the results did not clearly bear on the hypotheses relating 
to the treatment of errors when they occur and the procedure for defining 
the error criterion. 

Nevertheless, the Freeze/Reset condition did result in smaller errors 
in the final approach so ttiat a modified hypothesis, that the magnitude of 
errors permitted in training will affect learning, can be examined by 
contrasting the probe performances of the Freeze/Reset groups with the probe 
performances of the Conventional and Freeze/Flyout groups. This hypothesis 
varies slightly from the original' one that postulated effects of 
differential treatment of errors when they occurred. 

The notion that' learning will proceed more quickly if students can 
practice with fewer or smaller errors has* intuitive appeal. As noted 
previously in this report. Holding (1970) has suggested that frequently 
committed errors could become embedded in a student's response repertoire 
and thereby impede progression to asymptotic behavior. Lintern and Roscoe 
(1980) have suggested that training might benefit from manipulations that 
converge quickly on desirable control behavior. In addition, a criterion- 
setting process might be postulated where students who become used to 
performing with few errors endeavor to acheive similar standards even under 
more difficult conditions. Gi.ven the popularity of these notions, it seems 
noteworthy that there is n'o suggestion in the data that the reduced errors 
during Freeze/Reset training resulted in any differential rate of learning 
or terminal level of probe performance. While siome instructional strategies 
that promote more accurate learning behavior do assist skill acquisition 
(e.g., Lintern, 1980)^ it would appear that they do so by means other than 
their error-limiting function. 

There is some evidence, albeit insubstantial, that "freeze" had some 
adverse effects on performance during probe trials; in particular, that 
subjects training under the "freeze" conditions exhibited higher control 
activity. Pilot comments support the v-iew that the "freeze" may be aversive 
and that some applications of the "freeze" (e.g., the Fheeze/Reset) may 
change the nature of students' motivation in performing the task. That one 
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or more of these processes can affect perceptual-motor acquisition is 
indicated by the work of Payne and his associates (Payne, 1970; Payne and 
Art ley, 1972; Payne and Dunman, 1974; Payne and Richardson, 1972)* Thus 
while there js no evidence that the use of "freeze" can facilitate learning, 
there is some suggestion that it can disrupt the integrity of the task and 
thereby impede learning oh some aspects of the skill. In surfimary, the 
implications of the data for instructional procedures are -that attempts to 
enhance performance during learning will not necessarily facilitate skill 
acquisition, while injudicious use of the "freeze" function may disrupt it. 

From a methodological standpoint, the study is significant In that it 
supports the use of the probe technique as an alternative to the more 
traditional transfer-of-training methodology in the preliminary 
investigation of instructional treatment effects. In the present case, the 
probe technique proved to be sensitive to learning effects as well as to 
subtle performance differences which transferred from training trials to the 
subsequent probe (criterion) trials. 

The main methodological issues explored by examining the probe data 
were in relation to possible disruptive effects from frequent transfer 
between training and control conditions. There was no evidence of any 
disruptive effects, although this conclusion must be tempered with the 
observation that learning effects were minimal. The probe methodology could 
be valuable in a learning experiment, and its further examination with'data 
that show a strong learning trend would be useful. Stability of the probe 
trials was similar to stability of similarly located control trials. The 
question of how many probe trials are necessary might best be answered by a 
power analysis (Hays, 1963) and would require data that show some worthwhile 
differences between groups. Nevertheless this preliminary analysis of the 
probe technique suggests that it could be useful in future studies of skill 
acquisition. 
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SECTION V 
RECOMMENDATIONS 



Caution should be exercised when using the flight simulator's "freeze" 
feature duntag the performance of a continuous control task such as that 
involved the approach to landing task. Other tasks to which this advice 
might al^o apply are aerial refueling training and weapons delivery training. 

The/f act that an instructional technique may result in better trainee 
performahce during training should not be the sole criterion for its 
implementat^ion. While an effective instructional, technique can be expected! 
to aid and improve learning performances,- better performance on an 
instructional task does not necessarily lead to improved performance on the 
criterion task. 

The probe methodology is recommended as an alternative to the 
traditional transfer-of-training paradigm, especially for exploratory 
studies where training effectiveness may vary not only as a function of 
instructional approach but also as a function .of amount of training. 
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APPENDIX A. 
RESULTS/DATA SUMMARY TABLES 



TABLE Al. GLIDESLOPE RMS ERROR (IN FEET): 
^MEANS, STATISTICAL RELIABILITIES (*:p<.05, **:p<.01), 
AND VALUES OF ETA SQUARED (n^): TRAINING TRIALS ' 



Distance From 
the Ramp (Ft) 


6000 


- 4500 


',4500 


- 3000 


3000 


- 1500 


1500 


- 0 


Means 


Day 


NighT 


T)ay 


Night 


Day 


Night ^ 


Day 


Night 


CONDITIONS 


















Freeze/Flyout 
Di^placeiiient 


15.3 


14.2 


13.2 


13.5 


11.2 


10.7 


8'. 9 


9.3 


Frqeze/Flyout 
DiSD & Rate 


13.0 


13.5 


14.4 


11.5 


10.2 

• 


9.1 


8.1 


8.0 


Conventional 


16 .4 


14.7 


14.2 


15:4 


10.5 


12.9 


7.1 


11.9 


Freeze/ Reset 
Displacement 


13.1 


' 9.7 


11.0 


11.3 < 


8.6 


7.2 


4.9 


4.9 


Freeze/ Reset 
Disp & Rate 


8.6 


9.1 


7.7 


8.8 


6.7 


6.6 


4.1 


4^2 


TRAINING TRIALS 


















1-4 


14.5 


14.3 


14 ..5 


14.1 


12.4 


n.4 


8.5 


11.0 


7-10 


44.1 


12.1 


; 10.8 


10.8 


8.4 


8.8 


6.3 


7.1 


13-16 


13.2 


11.4 


13.3 


11.8 


9.2 


8.4 


5.9 


6.3 


19-22 


12.1. 


11.1 


10.6 


11.5/ 


■ 8.4 


8.9 


6.4 


6.5 



Reliabilities 
and 


P 




- P 




P 




P 


CONDITION 
















Day 


** 


.078 


* 


.079 


* 


.055 


* .118 


Night 


.053 


.077 


** 


.023 








TRAINING 
















Day , 






** 


.032 


** 


.055 


** .030 


Night 










* 


.023 




CI 
















Day 
















Night 
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TABLE A2. ANGLE OF AHACK RMS ERROR (IN AOA UNITS): 
MEANS, STATISTICAL RELIABILITIES C*:p<.05, **:p<.01), 
AND VALUES OF ETA SQUARED (n^): TRAINING TRIALS 



Distance Froni 
the Ramp (Ft) 


6000 


- 4500 


■ 4500 


- 3000 


3000 


- 1500 


1500 


- 0 


Means 


Day 


Night 


Day 


Night 


Day 


Night 


Day 


Night 


CONDITIONS 


















Freeze/Flyout 
Displacement 


.576 


-.425 


.778 


.580 


= 733 


.573 


1.067 


.845 


Freeze/Flyout 
Disp .4 Rate 


.415 


.358 


.653 


.661 


.590 


.579 


.795 


.706 


Conventional 


.370 


.355 


.482 


-.437 


.423 


.407 


.695 


■ .701 


Freeze/ Reset 
Displacenient 


.597 


.420 


.697 


.617 


.661 


.511 


.736. 


.675 


Freeze/' Reset 
Disp & Rate 


.508 


.374' 


.587 


.494 


.572 


.493 


.673 


.554 


TRAINING TRIALS 


















1-4 


.562 


.476' 

./ 
.'389 


.727 


.634 


.671 


.591 


.905 


.829 


7-10, 


.491 


.623 


.565 


.570 


.561 


.780 


.693 


13-16 


.463 


.363 


.583 


.,520 


.574 


.467 


.727 


.646 


19-22 


.436 


r322- - 


^ .623 


.473 


.560 


.427 


' .792 


.622 




Reliabilities 
and 


P 




P 




P 




P 




CONDITION 


















Day 


















Night 


















TRAINING 




\ 














Day 


** 


\022 


* 


.023 


* 


.019 


.053 


.021 


Night 


** 


.033 


** 


.035 


** 


.040 


** 


.035 



CT 

Day 

Night - ** .078 ** ^ .076 
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TABLE A3. AVERAGE ELEVATOR STICK ACTIVITY (IN UNITS/SEC 
WHERE ,RANGE OF CONTROL DISPLACEMENT IS FROM -1 TO +1 UNITS) : 
MEANS, STATISTICAL RELIABILITIES (*:p<.05, **:p<.01), 
, AND VALUES OF ETA SQUARED (n^): TRAINING TRIALS 



Distance From 
tne Kamp vrt; 


6000 


- 4500 


4500 


- 3000 


3000 


- 1500 


1500 


- 0 


Means 


Day 


Night 


Day 


Night 


Dav 


Night 


Dav 


Night 


titty T T T r» k\0 

CONDITIONS 






t 










■v. 


Freeze/Flyout 
Displacement 


.108 


• 

.068 


.110 


.070 


.128 


.088 


.202 


.1-32 


Freeze/Flyout 
Disp i Rate 


.097 


.085 


.098 


.085 


. 107 


.090 




.110 


Conventional 


.071 


.086 


.067 


.083 


.075 


.089 


.127 


.141 


Freeze/ Reset 
Displ acement 


.154 


.063' 


.142 


.063. 


.210 


.070 


.299 


.089 


Freeze/ Reset 
Disp i Rate 


.129 


.087 


.139 


.083 


.184 


.089 


► .324 


.104 


TRAINING TRIALS 


















1-4 


.143 


.079 


.134 


.078 


.169 


.090 


. .253 


.127 


7-10 


.103 


.078 


.106 


.079 


.135 


.087 


.220 


.120 


13-16 . 


.100 


.077 


.098 


.076 


.123 


.085 


.191 


413 


19-22 


.091 


.079 


.096 


.076 


.117 


.083 


.188 


.106 




Reliabilities 
and 


P 




P 




P 




P 




CONDITION 


















Day . 


















Night 


















TRAINING 


















Day 




.044 


* 


.029 


** 


.023 


** 


.015 



Night 

CT 



Day-. 
Night 
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TABLE A4. AVERAGE THROHLE ACTIVITY (IN UNlTS/SEC 
WHERE RANGE OF THROTTLE DISPLAGEMENT IS FROM 0 TO +1 UNITS) 
MEANS, STATISTICAL RELIABILITIES C*:p<,05, **:p<,01), 
I AND VALUES OF ETA SQUARED (n^): TRAINING TRIALS 



Ui Stance rrOm 

the Ramp .(Ft) 


6000 


- 4500 


4500 


- 3000 


3000 


- 1500 


1500 


- 0 ■ 


Means 


Day 


Night 


Day 


Night, 


Day ' 


Night 


Day 


Night 


CONDITIONS 


















Freeze/Flyout 
Displacement 


• LLC 


. Uoi 


f\A 1 


ni n 
.UiU 


DA 0 


.UIj 


.Uoj 


.yjco 


Freeze/Flyout 
Disp & Rate 


.081 


.083 


.009 


.011 


.011 


.013 


.024 


.025 


Conventional 


.081 


.081 


.008 


.008 


.010 


.009 


.019 


.Q22 


Freeze/ Reset 
Displacement 


.095 


-.080 


.024 


.009 


.022 


.010 


.033 


.025 


Freeze/ Reseat 
Disp &-R^ite 


,086 


.081 


.012 


.010 . 


.■015 


.011 


.023 


.016 


TRAINING TRIALS 


















1-4 


.087 


.081 


.012 


.069 


.015 


.011 


. .031 


.022 


7-10 ■ . 


.087 


.081 


.01-5 


.010 


.017 


.012 


.030 


.024 


13-16 . \ 


.096 


.080 


.026 


.009 


.026 


.010 


.0*35 


.022 


. 19-22 


.094 


.082 


.023 


.010 


.024 


.012 


.036 


.023 




Reliabilities 
and 


P 




P 




P 




P 





CONDITION 

Day 

Night 
TRAINING 

Day 

Night 

CT : 

Day 

Night * .071 
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TABLE A5. AVERAGE. AILERON STICK ACTIVITY (IN UNITS/SEC 
WHERE RANGE OF CONTROL DISPLACEMENT IS FROM -1 TO +1 UNITS) 
MEANS, STATISTICAL RELIABILITIES (*p<-Q5, **:p<.01), 
AND VALUES OF ETA SQUARED (n^): TRAINING TRIALS 



Distance From 
the Ramp (Ft) 


6000 - 


4500 


4500 


- 3000 


3000 


- 1500 


1500 


- 0 


M^Q nc 

rieans 


Ha \/ 

uay 


111 gnu 




IN 1 gnu 




n 1 gn U 


Haw 


n 1 (jil L 


CONDITIONS 


















Freeze/Flyout 
Di sdI arpmpnt 


.112 


.101 


.117 


.107 


.129 


. .134 


.168 


.146 


Frpp7p/Fl vout 
Disp & Rate ' 


.107' 


.094 


.113 


.093 


.122 


.099 


.136 


.123 


Conventional 


.071 


.083 


.073 


.090 


.079 


.103 


.100 


.112 


Freeze/ Reset . 
Displacement 


.102 


.079 


.106 


,082 


.118 


,102 


.145 


•.115 


Freeze/ Reset 
Disp & Rate- 


.099 


.110 


.103 


.117' ' 


.118 


.130 


.147 


. .130 


TRAINING TRIALS 






k 












1-4' 


.111 . 


.104- 


'" .111 


.108 


.132 


.128 


.163 


, .150 


7-10 


.099 


.098' ■ 


.101 


.104 


.110 


.119 


.133 


.127 


13-16 


.092 


.090 


.100 


.092 


.104 


.107 


.126 


.119 


lS-22 


.090 


.090 


.098 


.095 


.105 


-.112 


.132 


.112 




Reliabilities 
and 


P 




P 




P 




P 





CONDITION 
Day 
Night 

TRAINING 
Day 

• Night 



.032 



.047 



CT 



Day 
Night 



.053 .009 
" ' .050 



** 
** 



.050 
.044 
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TABLE A6. AVERAGE RUDDER PEDAL ACTIVITY (IN UNITS/SEC . 
WHERE RANGE OF- PEDAL DISPLACEMENT IS FROM -1 TO +1 UNITS): 
MEANS, STATISTICAL RELIABILITIES (*:p<.05, **:p<.01), 
AND VALUES OF ETA SQUARED (nM": TRAINING TRIALS 



Distance From 
the Ramp (Ft) 


6000 


- 4500 


4500 


- 3000 


3000 


- 1500 


1500 


- 0 


Means 


Day 


Night 


Day 


Night 


Day 


Night 


Day 


Night 



CONDITIONS 



Freeze/Flyout 
Displacement 


.063 


.049 


.058 


.046 


.061 


.051 


.074 


.064 


Freeze/Flyout 
Disp i Rate 


.059 


.058 


.056 


.057 


.057 


.060 


.059 


.061 


Conventional 


, .046 


.062 


.041 


.058 


^ .041 


.060 


.045 


.063 


Freeze/ Reset 
Displacement 


■ .060 


..040 


.054 


.034 


.053 


.036 ' 


.059 


* .038 


Freeze/ Reset 
Disp & Rate 


.058 


.061 


.056 


.056 


.057 


.057 


.060 


.056 



TRAINING TRIALS 



1-4 


.055 


.054 


.050 


.049 


.054 


.052 


.061 


.057 


7-10 


.057 


.056 


.052 


.052 


.053 


.053 


.059 


.059 


13-16 


.060 


.053. 


.057 


.050 


.054 


.054 ' 


.059 


.056 


19-22 


.056 


.056 


.052 


.053 


.053 


.055 


.058 


.059 





Reliabilities 
and p 




P 




P 




p 





CONDITION 
Day 
Night 



TRAINING 
Day 
Night 

CT 

Day 
• Night 
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TABLE A7. 6LIDESL0PE RMS ERROR (IN FEET): 
MEANS. STATISTICAL RELIABILITIES .C*:p<". 05, **:p<.01), 
AND VALUES OF ETA SQUARED Cn^): PROBE TRIALS 



Distance From 
the Ramp \Ft) 


6000 


- 4500 


4500 


- 3000 


3000 


- 1500 


. 1500 


- 0 




Day 


Night 


Day 


n 1 gn u 


uay 


n 1 gnt 


uay 


111 gnt 


CONDITIONS 


















Freeze/Flyout 
Displacement 


14.8 


14.2 


10.8 


12.7 


10.0 


9.3 


- 7.6 


7.4 


Vv^fi^y^ /Fl vniit" 
r 1 cc«>c/ r ijr uu u 

Oisp & Rate 


13.0 


12.8 


U.l 


11.1 


8.9 


9.4 


6.6 


9.7 


Conventional 


12.8 


15.0 


11.3 


12.5 


8.3 


10.1 


6.1 


8.-4 


Freeze/ Reset 
Displacement 


12.9 


11.0 


14.9 


14.2 


/ 10.9 


10.7 


7.8 


8.1 


Freeze/ Reset 


13.3 


11.1 


11.8 












Disp &.Rate 


9.9 


8.1 


8.8 


7.6 


7.9 


P,ROBE TRIALS 
V. ^5-6 


















14.7 


14.9 


12.3 


12.9 


9.8 


10.6 


9.3 


'9.6 


^ 11-12 


14.3 


11.7 


11.6 


11.7 


9.0 


10.1 


6.3 


8.6 


17-18 


13.1 


12.3 


13.0 


11.3 


. 10.0 


8.8 


6.7 


7.3 


23-24 


11.3 . 


12.3 


11.1 


12.4 


8.1 


9.2 


'6.2 


7.8 




Reliabilities 


















and 


P 




P 




P 




P 




CONDITION 

Day 

Night 
PROBE 

Day ' 














** 


.082 . 



Night 
CP ^ N 
Day 
Night 
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TABLE A8. ANGLE OF AHAGK RMS ERROR 
MEANS, STATISTICAL RELIABILITIES (*:p<.05, **;p<.01), ■ 
AND VALUES OF ETA SQUARED (t]^): PROBE TRIALS 



Distance From 
the Ramp (Ft) 


6000 


- 4500 


4500 


- 3000 


3000 


- 1500 


1500 


- 0 


Means 


Day 


Night 


Day 


Night 


Day 


Night ' 


Day 


Night 


CONDITIONS 


















Freeze/Flyout 
Displacement 


59 


38 


* 61 


59 


76 s 


59 


1 07 


71 

* i ± 


Freeze/Flyout 
Disp & Rate 


.43 


.43 


.66 


.73 , 


.59 


.57 


.70 


.83 


Conventional' 


.35 


.31 


.45 


.39 


.45 


.39 


.60 


.61 


Freeze/ Reset 
Displacement 


,•51 


.43 


.11 


.11 


.63'. . 


.66 


.87 


.70 


Freeze/ Reset 
Oisp 4 Rate 


.55 


.42 


' .70 


.53 


.54 


,50 


.90 


.55 


PROBE TRIALS 


















5-6 


.53 


.40 


.67 


.65 


.67 


.57 


1.14 


.68 


11-12 


.51 


.42 


.66 


.59 


.57 


.58 


.72 


.74 


17-18 


.43 


.37. 


.62 


.58 ' 


.56 


.52 


.80 


.65 


23-24 


.47 


.37 


.62 


.58 


.56 


.49 


.65 


.65 




Reliabilities 


















and 


P 




P 




P 




P 




CONDITION 
Day 
Night 

PROBE 
Day 
















.073 



Night 

CP 



Day 
Night 
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TABLE A9. AVERAGE ELEVATOR STICK ACTIVITY (IN UNITS/SEC 
WHERE -RANGE OF CONTROL DISPLACEMENT IS FROM -1 TO +1 UNITS): 
MEANS, STATISTICAL RELIABILITIES (*:p<.05, **:p<.01), 
■AND VALUES OF ETA SQUARED (n^): PROBE TRIALS 



Distance From 
the Ramp (Ft) ' 


6000 


- 4500 


4500 


- 3000 


3000 


- 1500 


1500 


- 0 


Means 


Day 


Night 


Day 


Night 


Day 


Night 


Day 


Night 


CONDITIONS 


















Freeze/Flyout 
Displacement 


.099 


.071 


.108 


.070 


.126 


.082 


.195 


.12,1 


Freeze/Flyout 
Disp & Rate 


.101 


.089 


.104 


.090 


.11 


.099 


.169 


.147 


Conventional 


.066 


.073 - 


.065 


.074 


.071 


.081 


.132 


.130 


Freeze/ Reset 
Displacement 


.105 


.066 


.136 


.066, 


.179 


..077 


.282 


.115 


Freeze/ Reset 
Disp ;& Rate 


.139 


.089 


.'■147 


.087 


-.193 


.097 


.348 


.123 


PROBE TRIALS 


















5-6 


.117 


.074 


.143 


.075 


.164 


.086 


.269 


.127 


11-12 


.098 


.083 


.104 


.082 


.127 


'.094 


.225 


.14 


17-18 


.101 


.077 


.105 


.076 


.129 


.085 


.205 


.116 


23-24 


.093 


.077 


.096 


- .076 


.124 


-.084 


.2 


.126 




Rel ia.bil ities 
and 


P 


11=^ 


P 




P 


11=^ 


P 





CONDITION 



Day 
Night 
PROBE 

Day — — ~ .01 

Night 

CP 

Day — — — . ^- 

Night 
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TABLE AlO. AVERAGE THROTTLE ACTIVITY (IN UNITS/SEC 
WHERE RANGE OF THROTTLE DISPLACEMENT IS FROM 0 TO +1 UNITS): 
MEANS, STATISTICAL RELIABILITIES (*:p<.05, **:p<.01), 
AND VALUES ^F ETA SQUARED (n^ ) : PROBE TRIALS 



Distance From 
the Ramp (Ft) 


6000 


- 4500 


4500 


- 3000 


3000 


- 1500 


1500 


- 0 


Means 


Day 


Night 


Day 


Night 


Day 


Night 


Day 


Night 


CONDITIONS 


















Freeze/Flyout 
Displacement 


.103 


.081 


.035 


.010 


.038 


.013 


.053 


.025 


Freeze/Flyout 

Ulbp a Kate 


.081 


.080 . 


.008 


.008 


.009 


.010 


.024 


.027 


Conventional 


.079 


.079 


.008 


.007 


.009 


.009 


.016 


.017 


Freeze/ Reset 
Displacement 


.083 


.080 


.012 


.009 


.013 


.010 


.036 


.027 


Freeze/ Reset 
Disp J Rate 


.084 


.092 


.014 


.010 


.016 


.012- 


.027 


.020 


PROBE TRIALS 


















5-6 


.084 


.081 


.012 


.009 


.015 


.011 


.031 


.025 


11-12 


.086 


.090 


.015 


.01 


.016 


.012 


.032 


.024 


17-18 


.093 


.08 


.023 


.008 


.024 


.010 


.039 


.022 


23-24 


.082 


.079 


.011 


.009 


.013 


.010 


.024 


.022 




Reliabilities 



and p P P P 



CONDITION 








Day 








Night 








PROBE 








Day 








Night 




* 


.022 


CP 








Day 




/ ;; 




Night - 


















bo 
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TABLE All. AVEI^GE AILERON STICK ACTIVITY (IN UNITS/SEC 
WHERE RANGE OF CONTROL DISPLACEMENT IS FROM -1 TO +1 UNITS): 
MEANS. STATISTICAL RELIABILITIES (*:p<.05, **:p<.01), 
AND VALUES OF ETA SQUARED (n^): PROBE TRIALS 



Distance From 
the Ramp (Ft) 


5000 


- 4500 


450Q 


- 3000 


3000' 


- 1500 


1500 


- U 


Means 


Day 


Kim mU'^ 

Night 


Day 


Kim ^l*'^ 

Night , 


Day 


Nignt 


Day 


Nignt 


CONDITIONS 










• 








Freeze/Flyout 
Displacement 


.104 


.095 


.118 


.101 


.137 


.128 


\l62 


.145 


Freeze/Flyout 
Disn A Ratp 


.11 


.116 •» 


.117 


.123 


.125 


.142 


.133 


.158 


Conventional 


.068 


.085 


' .068 


.091 


.08 - 


.109 


.109 


.133 


Freeze/ Reset 
Displacement 


.089 


.096 


.096 


.102 


.115 


.125 


.147 


.160 


Freeze/ Reset 
Disp & Rate 


.097 


.110 


.105 


.120 


.113 


.134 


.14 


.140 


PROBE TRIALS 


















5-6 


.104 


.107 


.113 


.114 


.127 


.133 


.159 


.174 


11-12 


.091 


.109 


.098 


. 119 


.'ll3 


.146 


.138 


.156 


17-18 


.093 


.089 


.098 


.097 


.114 


.117 


.14 


.133 


23r24 


.088 


.096 


.094 


.098 


.103 


.115 


.115 


.125 



Reliabilities 
and 


P 




P 




P 




P 




CONDITION 


















Day 


















Night 


















PROBE 


















Day 


* 


.019 




.024 


* 


.026 


** 


.059 


Night 


* 


.023 




.031 


** 


.032 


* 


.044 


CP 


















Day 














* 


.063 


Night 
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TABLE A12. AVERAGE RUDDER PEDAL ACTIVITY CiN UNITS/SEC 
WHERE RANGE OF PEDAL DISPLACEMENT IS FROM -1 TO +1 UNITS): 
MEANS, STATISTICAL RELIABILITIES C*:p<.05, **:p<.01), 
AND VALUES OF ETA SQUARED (n^): PROBE TRIALS 



Distance From 
the Ramp (Ft) 


6000 


o 

- 4500' 


4500 


- 3000 


3000 


- 1500 


15U0 


- 0 


Means 


Day 


^ Night 


Day 


Night 


Day 


Night 


Day 


Night 


CONDITIONS 


















Freeze/Flyout 
Displacement 


.057 


.051 


.057 


.045 


.065 


.051 


.073 


.064 


Freeze/Flyout 
Disp & Rate 


.058 


.062 


.056 


.057 


. .057 


.062 


.056 


.067 


Conventional 


.042 


.054 


.038 


.052 


.038 


.052 


.046 


.059 


Displacement 


.051 


.042 


.047 


.037 


.048 


.040 


.055 


.044 


Freeze/ Reset 
Disp & Rate 


.058 


.062 


.056 


.058 


.058 


.058 


,062 


.060 


PROBE TRIALS 
















V 


5-6 


.053 


.055 


.052 


.049 


.058 


.053 


.066 


.067 


11-12 


.053 


.056 


.051' 


.052 


.053 


.056 


.059 


.059 


4 17-18 


.056 


.054 


.051 


.05 


.053 


.054 


.057 


.056 


23-24 


.051 


.051 


.049 


.048 


.049 


.049 


.052 


.051 


Reliabilities 
and 


P' 




P 




P 









CONDITION 



Day 
Night 
PROBE 
Day 
Night 

CP 

Day 
Night 



46 5<; 
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APPENDIX B 

MEANS AND STANDARD DEVIATIONS OF 
•INDIVIDUAL ITEMS OF THE QUESTIONNAIRE; 
FREEZE/RESET AND FREEZE/FLYOUT CONDITIONS. 
SYMBOLS: FREEZE/FLYOUT = 0; FREEZE/ RESET = A 



1) Use of the freeze ifeature may be used to significantly decrease the 
overall training time required to learn the landing task. 



1- 



Strongly 
Di sagree 



'J 
c 




Strongly 
Agree 



2) Regaining control of the simulator following a freeze significantly 
added to the difficulty of the flying task in the simulator (when 
responding, consider each of the following phases of the maneuver 
separately): 

(a) ."IN THE MIDDLE" (first 1/3) 



Strongly 
Disagree 




J = 2.6 
a = 1.07 



(b) "IN THE GROOVE" (second 1/3) 

0^ 

■ 1 2 

Strongly • MefutraT 
Disagree * _ ^y"^ _ 

X =^.1 X = 3.1 

a = 1.1 a = 1.1 

(c) "IN CLOSE" (5-10 seconds from, the ramp) 



Strongly 
Disagree 



X-= 3.7 
a = 1.49 



X = 4.2 
o = .92 



Strongly 
Agree 



Strongly 
Agree 




Strongly 
Agree 
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(d) "AT THE- RAMP" 




Strongly 
Disagree 



Strongly 
Agree 



3) It was significantly easier to .attend to the ISO's feedback while 
frozen than while trying to listen and fly the .aircraft at the same time, 



Strongly 
Disagree 




5 

Strongly 
Agree 



4) Improvements in performance were highly correlated with a decrease 1n 
the number of freezes. 



Strongly 
Agree , 



Strongly 
Disagree 




X = 3.&^ 
a = .85 



5) Using the freeze feature to explicitly identify pilot errors during 
the "training" trials made it easier to detect errors on "test" trials 
whfen no feedback was given and when no freezes were in effect. 



Strongly 
Disagree 




Strongly 
Agree 



X,= 3.4 
a = 1.35 



X = 3.5 
a = 1.08 



6) Compared .with the usual practice of giving detailed feedback at the 
conclusion of a task, providing feedback immediately following an error is 
more effective. * ^ 



1- 



Strongly 
Disagree 



I = 3.3 

a = 1.16 




Strongly 
Agree 



X = 3.5 
o = .97 



7) In learning the task, I was more motivated by trying to avoid a freeze 
than by trying to fly the task correctly. 



1—. 

Strongly 
Disagree 



Neutral 



X = 2.7 
a ^ 1.42 
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8) The occurrence of the freeze was "frustrating" early in training, 



1- 



Strongly 
Disagree 



1= 3.5 
0 = 1.39 




J = 4.3 

0 = L06 



Strongly 

^_Agr,ee 



9) A helpful feature would be to present a "warning" signal (such zs 
an auditory , tone) prior to freezing the visual system. 



Strongly 
Disagree 




Strongly 
, Agree 



X = 3.5 
0 = L58 



10) Night approaches were more difficult to learn than the Day approaches, 



1- 



Strongly 
Disagree 



Neutra 




Strongly 
Agree 

X = 2.6 X = 3.0 ^ 

0^1.35 0'= 1.05 

11) Errors were more difficult to detect during the Night approaches than 
during' the Day approaches. 



Strongly 
Disagree 




1= 3 
0 - 1.15 



Strongly 
Agree 



12) '^Errors serve little purpose, since students may actually learn the 
errors that they commit." 



1-— " 2 4- 



Strongly 
Disagree 




Strongly 
Agree 



X 2.4 
0 = 1.07 



1-3-)- "Erirors help th§. student to focus on the critical elements bf task 
performance. " 



Strongly 
Disagree 



I " 3 

^^^JleutfaT^ 

X = 3.7 X = 4'.1 

0 - .48 0 = .57 




Strongly 
. Agree 
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14) "Instructional methods that allow errors to freely occur are ineffi- 
cient, since students spend valuable training time practicing incorrect 
responses." 



1- 



Strongly 
Disagree 



2.3 
1.06 




I = 2.8 
a = 1.03 



Strongly 
Agree 



15) "In committing errors, students learn how to recover from situations 
which at some later time may be caused not be task-specific errors but by 
conditions beyond their control (for example, by adverse weather, visi- 
bility, turbulence, etc.)." 



Strongly 
Di^sagree 




Strongly 
Agree 



16) "Pointing out 'errors' frustrates students, whereas pointing out what 
a student is doing^ 'right' is reinforcing." 



1- 



Strongly 
Disagree 




Strongly 
Agree 



1?) "Correct perfprmance results from a process of eliminating errors and 
not from a process of shaping desired performance." 

1 2 -y<^3-A "4 5 



Strongly 
Disagree 



Neutral 



Strongly 
Agree 



X = 3.0 
a = .94 



X = 3.2 
a = 1.03 



18) "A student's recognition of what is considered correct is dependent 
upon his being able to recognize what is incorrect (that is, an error)." 



1— 

Strongly 
Disagree 



2- 
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